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ABSTRACT 

Advancements in man-machine interfaces and control technologies used in space telerobotics and 
teleoperators have potential application wherever human operators need to manipulate multi-dimensional 
spatial relationships. Bilateral six degree-of-freedom (6-DOF) position and force cues exchanged between the 
user and a complex system can broaden and improve the effectiveness and intuitiveness of several diverse 
man-machine interfaces. 


A LOOK AT JPL MAN/TELEROBOT INTERFACES 

JPL has developed several man- machine interfaces that capitalize on the primary human communication 
channels, visual, tactile, and auditory, to transfer information to and from a telerobot. 

Much of the information fed back to the operator is visual. An operator’s control station, like those 
shown in Figures 1 and 2, typically provides multiple TV views of the remote worksite and may include a 
stereoscopic view as well for better depth perception. 

A computer generated graphics display provides the operator with force/torque vectors representing the 
contact interactions between the telerobot manipulator and it’s environment. There are also various other 
menu driven graphical interfaces that allow the system programmer, or even the operator, to easily adjust a 
broad spectrum of system parameters, such as gains, pot and encoder biases, etc., to accommodate personal 
preferences or system stability requirements. There may also be advanced multi-window graphical displays for 
implementing and pre-checking high level Al/Task Planning activities [1,2,3]. 

There may be situations where there are excessive time delays between the commanded input to telerobot 
and it’s actual occurrence. For example when the operator and the remote manipulator is separated by 
exceptionally long distances or when there are excessive computational complexities. In these circumstances 
the operator control station may be equipped with high speed predictive displays where graphical overlays of 
"phantom robots" are superimposed over the actual video images returned form the remote worksite. These 
phantom graphic images are updated in real time and show the impeding motions ahead of time and can help 
reduce confusion [4]. 

There are also voice recognition systems, standard computer terminals, key boards, mice and joysticks, and 
numerous lights, buttons, switches, alarms, buzzers, and beeps to keep track of. 

In the midst of all this, the operator is furnished with a unique and highly intuitive tool that provides him 
immediate and direct control of the remote telerobot manipulator. This device is known as a force-reflecting 
hand controller and will be the primary focus of this paper along with several of it’s potential spin-off 
applications. 


JPL HAND CONTROLLERS 

By simply grasping the handgrip of the hand controller and moving it in the desired direction, the 
operator has direct control of the remote manipulator’s position, and/or speed and direction. If the arm 
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comes in contact with some object at the remote worksite the resulting forces and torques are transmitted 
back through control system and the hand controller mechanism to the human operator’s hand thereby 
allowing him to sense the remotely applied forces. The complex transformations and servo control loops 
required to perform such an operation are all performed completely transparent to the human operator. e 
can perform the remote task almost as if he were performing it in person. Many experiments have shown that 
when a human operator is provided with force feedback information the task completion tunes and extraneous 
contact forces are greatly reduced [5]. 

t winning in the late 1970’s, JPL developed a series of FRHC’s which have been installed in several 
research projects over the years including JPL’s Telerobot Testbed Facility and at the Advanced Teleoperator 
Research Lab (shown in Figures 1 and 2 respectively). These early JPL hand controllers, dubbed the Models 
A, B, and C, have collectively undergone many hundreds of hours of testing and operation and have been the 
means of greatly expanding the scientific and engineering knowledge about man-machine interfaces as they 
apply to bilateral teleoperation of robotic manipulators. Extensive and sophisticated control methodologies 
have been developed based on these hand controllers [6,7,8]. 

In 1988 initial funding was provided to develop a more robust flight-qualifiable force-reflecting hand 
controller (FRHC). In addition to meeting the rigorous demands of space flight it was also to have a smaller 
stowage volume and much higher mechanical bandwidth than its predecessors. After several months o 
intense design and development a pre-flight prototype FRHC, called the Model X, was produced (see Figures 
3 and 4). The basic design, physical description, capabilities and the results of early testmg are fully described 

in[9]. 


BASIC FEATURES OF THE MODEL X FLIGHT HAND CONTROLLER 

The Model X FRHC has many outstanding features which combine to make it a substantial step forward 
in hand control design. Several of the features are unique to the Model X and promise to improve the 
performance bandwidth of the human operator in the control of remote applications. 

At first glance some features may not be immediately obvious or impressive to someone unfamiliar with 
the subtleties of hand controller design. Where possible, the importance of these, less obvious, features have 
been highlighted. Some of the features were measured experimentally while others values were derived 
analytically or approximated. 

1) Highly Intuitive Operation. 

The remote tasks are performed as if in the operator’s own body-reference frame. 
Teleoperation complexities are largely transparent to the operator. The design inherits 
sophisticated control modes from predecessors. 

2) Generalized Applicability. 

The Model X can be use to command diverse remote manipulator’s that are kinematically 
dissimilar to itself. 

3) Excellent 6-DOF Position and Orientation Resolution. 

With the Model X fully out stretched, in it’s worst-case position, it can detect and respond to 
positional inputs at the handgrip as small as 0.002 of an inch. 

4) Excellent Dynamic Force Range. 

In its present implementation, the prototype Model X can output feedback forces to the 
human operator ranging from = 3 to 67 oz. (0.2 to 4.2 lbs) when fully extended in 1-g 
(worst-case). When less than fully extended it can output forces as high as 12 lbs. Also, the 
force output levels in the zero-g of space would be substantially higher. 
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5) High Mechanical Stiffness and Fast Mechanical Response. 

The Model X is a much stiffer mechanism than other current hand controller designs with an 
approximate natural frequency of around = 25 hz. With this higher mechanical bandwidth 
the Model X can reproduce higher fidelity force feedback cues to the human operator, 
making the feedback more crisp and distinguishable. 

6) Flight-Qualifiable. 

The Model X has a rugged structural design which is necessary for the rigorous demands of 
space flight. 

7) Low Mass. 

The prototype Model X mechanism has a mass of only 27.6 lbs. And further weight 
reduction is possible. 

8) Small Stowage Volume. 

When not in use the Model X can fold up and stow in a relatively compact case measuring 
9.6 x 17 x 20 inches (1.9 ft 3 ) 

9) Large Workspace. 

At over 4 ft 3 the Model X has a much larger work envelope than previous FRHC designs. 
Reduced indexing required. 

10) Range of Motion Optimized to Human. 

The Model X’s work envelope overlaps much of the natural reach of the person operating the 
hand controller. The mechanical layout is such that it shadows the movements of the human 
operator and does not interfere with the viewing and operation of other control station 
equipment. 

11) Low Friction. 

It has relatively low friction levels ranging from 3.0 oz. to 6.0 oz. within target areas of the 
work envelope. 

12) No Backlash. 

The Model X design uses a unique cable/pulley transmission system that eliminates all 
backlash. No gears are used. 

13) Low Inertias. 

The Model X has a unique design that permits all six motors, constituting the greatest 
portion of the mech anis m’s mass, to be mounted at or near the hand controller’s base thus 
greatly reducing the effects of inertia that may otherwise be detrimental to the Hand 
Controller’s useful operation. 

14) Highly Backdrivable. 

15) No External Counter Balancing Required. 

16) Fully Decouple Joint Motion. 

Each of the six joints on the Model X rotates independently from all others eliminating 
additional computations otherwise required. It also reduces the complexity of dynamic and 
friction compensation techniques. 

17) Simple and Computationally Fast Kinematic Solutions. 

The individual link coordinate reference frames either intersect at right angles or are parallel 
to one another. This makes many of the terms in the coordinate transformations reduce to 
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either 1 or 0 and thereby greatly simplify their computations. It also has no link offsets or 
link twist angles to factor in. 

The foregoing list of features provides many indications that the Model X hand controller will be a viable 
and effective user interface to spatially dynamic multi-degree-of-freedom systems. 

SIX POTENTIAL SPIN-OFF APPLICATIONS FOR THE MODEL X HAND CONTROLLER 

The Model X hand controller is a highly intuitive, highly versatile human interface to dynamic 
multi-degree-of-freedom machines. As a high resolution 6-DOF position input device with exceptional force 
feedback capabilities it may prove to be a useful human interface in applications other than the control ot 
teleoperated robotic arms. The Model X has been optimized to fit the human operator’s range of motion. As 
such, it has the potential of being the fundamental interface that a human operator needs m order to 
manipulate multi-dimensional spatial relationships where force cues can help associate a coordinated response. 

SINGLE-HAND FLIGHT CONTROLLER FOR HELICOPTERS 

Helicopters are undoubtedly one of the most difficult and challenging vehicles to pilot, especially 
Hnring the high agility maneuvers required in modern combat. Many research efforts have been 
undertaken to reduce and optimize the workload demanded of military helicopter pilots [10, 11, 12J. 

One possible spin-off application of the Model X force-reflecting hand controller technology, may be in 
the field of helicopter pilot flight control interfaces. Advances in digital flight control techniques and 
fly-by-computer technologies are opening the door for sophisticated new pilot/vehicle interfaces that can 
potentially significantly simplify flight path management and thereby allow the pilot to concentrate more on 
the overall mission management objectives. 

As a flight control interface, the Model X hand controller would allow the pilot to command even the 
most complex helicopter flight path using only one hand, whereas the four conventional pilot inputs generally 
in use today require the use of both hands and both feet (see artists concept Figure 5). 

The Advantages include: a) Significant Reduction in workload through highly intuitive smgle-hand fhght 
control; b) decreased cockpit congestion; c) reduced training time; d) improved maneuverability and hand g 
qualities; e) highly adjustable performance attributes; 0 easy system upgrades and modifications; g) 
accommodates individual pilot preferences and modular mounting configurations; h) seamless auto/manual 
mode transitions; i) decreased vulnerability and complete redundancy possible; j) lighter weight, an , ) ess 
expensive than conventional controls. 

REMOTELY CONTROLLED UNDERWATER ROBOTS 


Without a doubt, the wave of the future in ocean engineering and science are remotely controlle 
unmanned vehicles. As a highly intuitive and compact man/machine interface, the Model X hand controller 
has multiple potential applications within the field of remotely operated vehicles (ROV s). 


In Recent years we have seen the emergence and evolution of several new underwater research vehicles 
like Woods Hole’s "Jason", Eastport’s "Gemini 11", Hydrovision’s "HybalT, and NOSC’s "Advanced Tethered 
Vehicle” Some of the more advanced remotely operated vehicles (ROV’s) can be maneuvered in 6 ( l - •* 

forward/back, up/down, side-to-side, roll, pitch, and yaw) and include sophisticated navigation modes such as 
auto-heading, auto-depth, and position hold. 


The ROV’s often carry an array of sensors and tooling such as: still and video cameras, lights, scanning 
sonar, rate gyro’s, temperature sensors, salinity sensors, etc. In addition, some unmanned submersibles may 
also employ one or more robotic manipulators and grippers (see Figure 6). 
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There are three areas in which the Model X force-reflecting hand controller might conceivably be used to 
enhance the man/machine interfaces of advanced underwater exploration robots. These areas are: (1) remote 
piloting of the unmanned, underwater robotic vehicle; (2) remote control of the robotic manipulators mounted 
on the vehicle; and, (3) remote control of camera pan, tilt, zoom, and focus. All integrated into a single hand 
controller. 


NUCLEAR INDUSTRY REMOTE MATERIALS HANDLING 

Since the 1940’s the nuclear industry has developed and used a wide variety of hand controllers. Many of 
the recent advancements in teleoperation and telerobotics can be directly attributed to research efforts in this 
field. The nuclear industry, both private and military, employs most of the hand controllers in used today 
which are typically mechanically coupled master/slave type devices. 

The nuclear industry has many applications where teleoperation is essential both from a safety and an 
economics standpoint. Clean-up or even routine assembly and maintenance operations are composed of tasks 
that are far too complex to be performed by unsupervised autonomous robots and too dangerous to be done 
by human workers. Because of these issues the industry has turned to a legion of remotely operated devices. 

A single, generalized hand controller, like the Model X, could be used to control an u nlimi ted variety of 
remote manipulation devices that differ substantially in both structure and kinematics. Devices like overhead 
cranes, spanning hundreds of meters, used to move equipment or pallets of radioactive material; or small 
dexterous robot manipulators used inside hot cells to assemble fuel canisters. The operator control stations 
for these various tasks could all utilize a common type of man- machine interface. 

FLIGHT CONTROL OF REMOTELY PILOTED VEHICLES (RPVs) 

RPV’s are typically envisioned as high-tech sentinels, pinpoint target acquisition instruments, 
aircraft/missile decays, and covert surveillance platforms [13]. 

Most experimental RPV’s are designed to fly like an airplane but some are capable of more complex flight 
trajectories more like a helicopter. They can hover, fly forward/back, up/down, side-to- side, and perform roll, 
yaw, and pitch attitude control maneuvers. Presently RPV flight paths are typically controlled by either 
pre-set programs or joysticks similar to radio controlled model planes. 

The Model X could be employed as the man-machine interface through which a human operator remotely 
pilots an RPV. The mechanics of this new type of pilot/vehicle interface would essentially be the same as that 
described for helicopter flight controls except that here the vehicle’s pilot would not be physically on the 
vehicle being flown. It could easily be made into a light weight and backpackable. 

SIX-DOF COMPUTER IMAGE MANIPULATION WITH FORCE FEEDBACK CUES 

Many advanced computer graphics packages allow the user to create three dimensional objects. These 
objects can then be manipulated relative to the user’s point of view and to each other. 

Today CAD programs often allow the user to zoom in, zoom out, and move graphical objects up, down, or 
sideways relative to the screen. A user can also rotate an object about an arbitrary axis. There have been 
several new devices that attempt to make this object manipulation more user friendly and less cumbersome, 
like 3-DOF joysticks, "advanced mice" and track balls [14], but they often limit manipulation to only one, two, 
or three dimensions at a time instead of the six required to pinpoint any object’s position and orientation (i.e., 
X, Y, Z, Roll, Pitch, Yaw). 

Using the Model X, computer users would be able to move and re-orient computer graphic images as 
easily and as intuitively as if they had reached out and done so with their own hand. It would be like having 
the functionality of six joysticks rolled into one. 
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Add to this the capability of the Model X to output feedback forces and one has a completely new level 
of computer interface. The user would then be able to "feel" objects as well. This would allow a wide range 
of new uses. For example, force cues could be given to the user when objects being manipulated on the 
screen contact one another, or there could be virtual buttons and switches having actual mechanical detentes, 
or the user might even sense the relative mass and inertias of objects. The capability for the computer to 
output forces could greatly expand the sensory bandwidth of the human interface at graphics workstations, 
creating a virtual environment with the added dimension of dynamic force. 

A more specific example might include research into new chemical compounds is being performed on 
computers. It is sometimes very beneficial to the researcher to be able to visualize complex chemical chains m 
3-D and then to manipulate and mate new chains at various locations by hand. As explained in the previous 
application the Model X has the potential of greatly enhancing the computer graphic interface The force 
feedback capability might be employed to provide the researcher with a sense of feel for the structure o 
microscopic substances he is working with and provide additional feedback about the connec vity o vario 
complex substances. Appropriate attractive and repulsive forces could be applied to particular molecu es. 

One researcher that studies the complexities and spatial relationships of DNA and other genetic materials 
has sought to augment the laborious process of checking potential DNA combinations and fits by using C 
rendered displays and a hand control device [15]. By adding the dimension of force feedback and the 
simulated molecular docking process might be enhanced even further. 

REDUNDANT DOF DEXTEROUS ROBOT MANIPULATORS 

There is a significant research effort under way within the robotics community to develop new methods for 
controlling redundant degree-of-freedom (DOF) manipulators, particularly robots with seven or more DOF. 

From extensive earlier work with 6-DOF manipulators (which is the minimum number of 
degrees-of-freedom necessary to provide complete position and orientation capability) we learned that several 
features inherent in the mechanism such as: kinematic simplicity, friction, manipulator stiffness, mass 
distribution, backlash, backdrivability, and others, where important issues to consider during the mechamca 
design of the robot. If they were not dealt with properly during the mechanism design then they could be 
much more cumbersome and/or impossible to compensate for later on in the software control algorithms. 

The technologies used in the design of the Model X hand controller could be used to design new 
redundant DOF manipulators that had excellent position and force control characteristics. 

SUMMARY 


Many engineers, designers and researchers in several fields have understood the need for better 
man-machine interfaces. Following an informed discussion on the state-of-the-art m teleoperation in a recen 
Scientific American article, Dr. William Uttal, professor of psychology at Arizona State University, concluded 
by making the following observation: 


"These prototype teleoperators show that most of the physical technology needed for these highly 
complex and useful teleoperators is already available. The miyor challenges now lie at the interface 
of human and machine. Until sufficient progress in computer technology makes it possible to 
substitute artificial intelligence for human intelligence, psychological insights mto the best ways to link 
human operators to machines will pace the evolution of remote workstations" [16]. 


The Model X FRHC is a significant step forward in hand controller design and may prove to be an 
extremely useful new man-machine interface in several diverse applications including: (a) new pdot/hehcopter 
flight control interfaces; (b) control of underwater remotely operated vehicles, manipulators, and other system 
components; (c) remote materials handling, assembly, maintenance, and inspection operations in the nuclear 
industry; (d) new pilot/RPV flight control interfaces; (e) 6-DOF computer image manipulation with force 


355 



feedback cues to the operator; and, (f) design of redundant DOF dexterous robot manipulators with good 
force controllability, minimal kinematic complexity. 
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Figure 1 The Advanced Teleoperator Research Laboratory 

Showing the Model B FRHC, three camera views, a force/torque 
display, and a predictive display. The remote telerobot and 
task board are visable in the background. 



Figure 2 Telerobot Testbed Facility 

The basic operator control station has five separate camera views, 
plus a stereoscopic view, two Model C FRHC's, a graphical multi- 
window user interface, and a voice recognition system. 
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Figure 3 The Model X Force-Reflecting Hand Controller 
Shown in the vertical mounting configuration. 



Figure 4 The Model X shown in the horizontal mounting configuration. 
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Figure 5 The Model X Hand Controller could potentially be used as a 

radically new 6-DOF pilot/helicopter flight control interface 
supporting advanced digital flight control systems. 



Figure 6 The Model X Hand Controller could be used to pilot this 

underwater vehicle and control it's robotic arms and cameras. 
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